Report  No.  DODIG-2014-006 


Inspector  General 

US.  Department  of  Defense 


Program  Management  Office 
Provided  Adequate  Oversight  of 
Two  Contracts  Supporting  the 
Defense  Enterprise  Accounting 


INTEGRITY  ★  EFFICIENCY  ★  ACCOUNTABILITY  ★  EXCELLENCE 


Report  Documentation  Page 

Form  Approved 

OMB  No.  0704-0188 

Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 

VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 

1.  REPORT  DATE 

25  OCT  2013  2' REPORT  TYPE 

3.  DATES  COVERED 

00-00-2013  to  00-00-2013 

4.  TITLE  AND  SUBTITLE 

Program  Management  Office  Provided  Adequate  Oversight  of  Two 
Contracts  Supporting  the  Defense  Enterprise  Accounting  and 

Management  System 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS (ES) 

Department  of  Defense  Inspector  General, 4800  Mark  Center 

Drive, Alexandria, VA, 22350-1500 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS  (ES) 

10.  SPONSOR/MONITOR’S  ACRONYM(S) 

11.  SPONSOR/MONITOR’S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

14.  ABSTRACT 

15.  SUBJECT  TERMS 

16.  SECURITY  CLASSIFICATION  OF:  17.  LIMITATION  OF 

ARSTRATT 

1 8 .  NUMBER  1 9a.  NAME  OF 

OF  PAGES  RESPONSIBLE  PERSON 

a.  REPORT  b.  ABSTRACT  c.  THIS  PAGE  Same  3S 

unclassified  unclassified  unclassified  Report  (SAR) 

16 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


INTEGRITY  ★  EFFICIENCY  ★  ACCOUNTABILITY  ★  EXCELLENCE 


Mission 

Our  mission  is  to  provide  independent,  relevant,  and  timely 
oversight  of  the  Department  of  Defense  that:  supports  the 
warfighter;  promotes  accountability,  integrity,  and  efficiency; 
advises  the  Secretary  of  Defense  and  Congress;  and  informs 

the  public. 

Vision 

Ourvision  is  to  be  a  model  oversight  organization  in  the  federal 
government  by  leading  change,  speaking  truth,  and  promoting 
excellence;  a  diverse  organization,  working  together  as  one 
professional  team,  recognized  as  leaders  in  our  field. 
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MEMORANDUM  FOR  ASSISTANT  SECRETARY  OF  THE  AIR  FORCE 

(FINANCIAL  MANAGEMENT  AND  COMPTROLLER) 

SUBJECT:  Program  Management  Office  Provided  Adequate  Oversight  of  Two  Contracts 
Supporting  the  Defense  Enterprise  Accounting  and  Management  System 
(Report  No.  DODIG-2014-006) 

We  are  providing  this  report  for  your  information  and  use. 

Our  original  objective  was  to  determine  whether  DoD  and  U.S.  Air  Force  officials 
administered  contracts  supporting  the  Defense  Enterprise  Accounting  and  Management 
System  (DEAMS)  in  an  effective  and  efficient  manner.  Specifically,  we  planned  to  determine 
whether  Government  oversight  of  contractor  performance  was  adequate,  quality 
assurance  plans  were  adequately  implemented,  contracts  were  funded  in  accordance  with 
applicable  regulations,  and  modifications  that  increased  costs  were  properly  supported. 
During  audit  planning,  we  revised  our  specific  objective  to  determine  whether  Program 
Management  Office  (PMO)  oversight  officials1  provided  adequate  oversight  of  contractor 
performance  and  implemented  quality  assurance  plans  for  two  contracts.  Therefore, 
we  reviewed  contract  documentation  from  2004  through  2012  for  the  system  integration 
(SI)  contract  FA8770-06-F-8001  and  the  independent  verification  and  validation  (IV&V) 
contract  FA8770-09-C-0064.  The  contracts  had  an  approximate  total  value  of  $109  million. 

Based  on  the  changes  to  the  specific  objectives  and  the  audit  results,  we  will  not  address 
the  overall  objective  related  to  DoD  and  Air  Force  officials’  administration  of  contracts 
or  the  remaining  specific  objectives  related  to  contract  funding  and  contract  modification 
in  a  follow-on  report.  Overall,  PMO  officials  provided  adequate  oversight  of  contractor 
performance  for  the  SI  and  IV&V  contracts  and  implemented  methods  for  documenting 
contractor  performance  on  both  contracts  that  met  the  intent  of  a  quality  assurance 
(surveillance)  plan.  We  considered  management  comments  on  a  discussion  draft  of  this 
report  in  preparing  the  final  and  revised  the  report  as  appropriate. 


1  DEAMS  PMO  oversight  officials  include  the  contracting  officer,  contracting  officer's  representative,  and  quality  assurance  program 
coordinator;  a  multifunctional  and  integrated  product  team;  and  other  personnel  assigned  to  oversee  DEAMS  contracts. 
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System  Is  Key  to  Financial  Statement  Auditability 

The  May  2013  Financial  Improvement  and  Audit  Readiness  Plan  stated  that 
Enterprise  Resource  Planning  Systems2  continue  to  be  an  essential  part  of  the 
Department’s  audit  readiness  efforts.  DEAMS  is  a  key  component  to  the  overall 
Financial  Improvement  and  Audit  Readiness  Plan  and  strategy  to  address  DoD 
financial  management  challenges.  DEAMS  is  a  cross-service  initiative  between 
the  Air  Force,  the  U.S.  Transportation  Command,  and  the  Defense  Finance  and 
Accounting  Service.  According  to  the  DEAMS  website,  its  mission  is  to  support  the 
warfighter  with  timely,  accurate,  and  reliable  financial  information  so  DoD  managers 
can  make  efficient  and  effective  decisions. 

In  DoD  Office  of  Inspector  General  Report  No.  DODIG-2013-111,  "Status  of  Enterprise 
Resource  Planning  Systems’  Cost,  Schedule,  and  Management  Actions  Taken  to 
Address  Prior  Recommendations,"  August  1,  2013,  we  reported  the  life-cycle  cost 
for  the  DEAMS  program  was  estimated  at  $2.1  billion  as  of  July  2012.  However,  this 
estimate  was  decreased  to  $1.9  billion  as  of  June  2013.  According  to  Government 
Accountability  Office  Report  No.  GAO-13-311,  "Selected  Defense  Programs  Need 
to  Implement  Key  Acquisition  Practices,”  March  28,  2013,  the  program  spent 
approximately  $334  million  over  9  years  before  establishing  its  first  Acquisition 
Program  Baseline  and  developing  a  robust  estimate  for  how  much  DEAMS  was 
expected  to  ultimately  cost.  Based  on  information  provided  by  DEAMS  personnel,3  we 
identified  a  universe  of  26  contracts  valued  at  approximately  $266  million,  in  support 
of  the  DEAMS  program,  including  SI,  IV&V,  assistance  and  advisory  services,  and 
licensing-type  service  contracts. 

Program  Management  Office  Implemented  the 
Enterprise  Resource  Planning  System 

According  to  the  DEAMS  Program  Charter,  the  PMO  consists  of  project  and  functional 
branches.  PMO  officials  are  responsible  for  delivering  a  configured  system  that 
meets  the  DEAMS  Functional  Management  Office  (FMO)  requirements.  The  FMO 
is  responsible  for  interfacing  with  DEAMS  stakeholders  and  defining  the  functional 


2  Enterprise  Resource  Planning  systems  were  designed  to  replace  numerous  subsidiary  systems,  reduce  the  number 
of  interfaces,  and  standardize  and  eliminate  redundant  data  entry  while  providing  an  environment  for  end-to-end 
business  processes. 

3  Information  provided  by  the  Program  Management  Office  and  Functional  Management  Office. 


DODIG-2014-006 


requirements  for  DEAMS.  The  DEAMS  Quality  Assurance  and  Management  function 
is  an  independent  PMO  function  that  reports  directly  to  the  program  manager  and 
deputy  program  manager. 

In  April  2005,  DEAMS  received  Milestone  A4  authority  to  develop  and  deploy  a 
technology  demonstration  to  Scott  Air  Force  Base,  Illinois,  and  its  various  tenant 
commands.  On  February  3,  2006,  PMO  personnel  awarded  the  SI  contract  to 
Accenture  Limited  Liability  Partnership,  now  Accenture  Federal  Services  (Accenture), 
to  reengineer  DEAMS  financial  business  processes  and  develop  and  implement  the 
Increment  I  Technology  Demonstration.5  The  SI  contract  was  a  firm-fixed-price, 
performance-based  incentive  award  that  included  contract  line  item  numbers  for 
reimbursable  travel  expenses  and  other  direct  costs  incurred  by  the  contractor.  PMO 
awarded  the  SI  contract  to  help  integrate  a  new  financial  management  system  to  ensure 
proper  handling  of  common  Federal  accounting,  reporting,  and  electronic  business 
transactions.  One  of  the  deliverables  for  the  SI  contract  was  an  end-to-end  process 
for  DEAMS  so  the  Air  Force  could  achieve  the  goal  of  a  clean  audit  opinion.  The  SI 
contract  expired  on  April  30,  2012.  As  of  May  10,  2013,  approximately  $106  million 
was  obligated,  and  the  total  amount  obligated  was  expended. 

On  September  30,  2009,  PMO  personnel  awarded  the  IV&V  contract  to  Ryan 
Consulting  Group,  Inc.  (Ryan  Consulting)  to  provide  the  PMO  and  oversight 
authorities  with  independent  evaluation  of  DEAMS  processes  and  product 
improvement  services.  The  IV&V  contract  was  a  time  and  materials  (labor  hours) 
contract  that  included  contract  line  item  numbers  for  reimbursable  travel  expenses 
incurred  by  the  contractor.  Ryan  Consulting  was  tasked  with  assisting  PMO  with 
monitoring  the  SI  contract  progress  by  verifying  that  the  SI  contract  project  phases 
complied  with  appropriate  entry  and  exit  criteria,  evaluating  system  testing  efforts, 
and  providing  accounting  and  finance  functional  expertise  to  ensure  the  system 
satisfied  all  applicable  requirements.  The  IV&V  contract  expired  on  December  22,  2011. 
As  of  January  31,  2012,  approximately  $2.7  million  was  obligated,  and  the  total  amount 
obligated  was  expended. 


4  Milestone  A  is  the  point  at  which  a  recommendation  is  made  and  approval  sought  regarding  starting  or  continuing  an 
acquisition  program  into  the  Technology  Development  Phase. 

5  Technology  development  is  a  phase  within  the  Defense  Acquisition  System  that  follows  a  Milestone  A  decision.  This  phase 
includes  refining  requirements  and  processes  until  the  system  achieves  the  required  maturity  level,  and  it  may  require 
multiple  development  increments. 
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Officials  Provided  Adequate  Oversight  of  Contracts 

PMO  officials  provided  adequate  oversight  of  contractor  performance  for  the  SI  and 
IV&V  contracts.  PMO  contracting  officers  designated  contracting  officer’s 
representatives  (CORs)  by  issuing  designation  letters  and  specifying  their  authority 
and  period  of  designation  for  the  SI  and  IV&V  contracts  in  accordance  with 
Federal  Acquisition  Regulation  (FAR)  Subpart  1.6,  "Career  Development,  Contracting 
Authority,  and  Responsibilities."  PMO  contracting  officers  were  unable  to  provide 
evidence  that  all  CORs  were  trained  in  accordance  with  the  Air  Force  policy  effective 
at  the  time  that  the  SI  and  IV&V  contracts  were  awarded.6  However,  PMO  contracting 
officers  ensured  that  CORs  were  properly  trained  in  accordance  with  Air  Force 
Federal  Acquisition  Regulation  Supplement  Mandatory  Procedure  5301.602-2(d).7 
The  CORs  for  the  SI  and  IV&V  contracts  used  PMO  officials  to  assist  them  in 
monitoring  the  performance  of  Accenture  and  Ryan  Consulting.  For  example,  integrated 
product  teams  (IPTs)  assisted  in  monitoring  the  SI  and  IV&V  contracts  and  helped 
refine  requirements  as  needed.  The  IPTs  consisted  of  functional  area  experts  who 
monitored  contractor  performance,  reviewed  deliverables  for  compliance,  and  provided 
technical  advice. 

PMO  officials  met  regularly  with  Accenture  and  Ryan  Consulting  to  discuss  the 
program  status  and  challenges  and  to  refine  requirements.  PMO  officials  periodically 
assessed  contractor  performance  during  performance  reviews  and  provided  feedback 
to  Accenture  and  Ryan  Consulting  in  the  form  of  incentive  payments8  and  annual 
assessments,  respectively.  In  addition,  the  PMO  program  manager  held  regular 
meetings  with  senior  Air  Force  officials  to  keep  DEAMS  stakeholders  informed  of  the 
program  status  and  identify  program  risks  and  other  program  metrics. 

Contractor  Performance  Documentation  Maintained 

PMO  officials  implemented  methods  for  documenting  contractor  performance  for 
both  the  SI  and  IV&V  contracts  that  met  the  intent  of  a  quality  assurance  surveillance 
plan  (QASP).  A  PMO  contracting  officer  approved  a  performance  plan  for  the  SI  contract 
on  February  28,  2011.  Although  PMO  contracting  officers  did  not  prepare  a  QASP  for 


6  Air  Force  Federal  Acquisition  Regulation  Supplement  Mandatory  Procedure  (MP)  5346.103,  August  2005,  required 
that  CORs  receive  Phase  I  and  Phase  II  training  prior  to  contract  award  and  required  the  contracting  officer  and  quality 
assurance  program  coordinator  to  maintain  documentation  of  Phase  I  and  Phase  II  training. 

7  The  Phase  I  and  Phase  II  training  requirements  outlined  in  MP5346.103  were  replaced  by  MP5301.602-2(d),  April  21,  2011, 
which  requires  16  hours  of  COR  specific  refresher  training,  annual  ethics  training,  and  any  Activity  required  training. 

8  PMO  officials  outlined  the  criteria  for  incentive  payments  for  the  SI  contract  in  an  award  fee  plan  and  performance-based 
incentive  plan.  Also,  PMO  officials  determined  award  fee  incentives  based  on  the  assessed  value  of  Accenture's  support  to 
the  DEAMS  program  and  performance-based  incentives  on  Accenture's  performance  of  SI  contract  functions. 
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the  SI  contract  in  conjunction  with  the  preparation  of  the  statement  of  work,  the 
performance  plan  identified  the  contract  elements  that  required  surveillance  and 
the  methods  used  to  provide  surveillance  as  required  by  FAR  Subpart  46.4, 
"Government  Contract  Quality  Assurance."  Specifically,  the  performance  plan 
provided  information  to  contractors  on  the  requirements,  level  of  performance 
expectations,  and  how  the  PMO  would  confirm  the  services  were  provided. 
The  performance  plan  identified  a  multifunctional  team  responsible  for  monitoring 
and  evaluating  Accenture’s  performance  on  an  ongoing  basis  and  required 
that  the  COR  perform  quarterly  assessments  of  Accenture’s  performance.  The 
performance  plan  also  outlined  the  performance  objectives  and  performance 
standards  by  which  Accenture  would  be  evaluated.  Based  on  the  content  of  the 
performance  plan,  we  determined  that  the  performance  plan  met  the  intent  of 
a  QASR 

The  performance  plan  for  the  SI  contract  established  the  use  of  assessment  logs 
to  document  the  timeliness  and  quality  of  deliverables  and  to  control  contract  cost. 
PMO  officials  engaged  in  other  ad  hoc  procedures  for  monitoring  contractors  and 
documenting  the  results  instead  of  maintaining  assessment  logs  or  performing 
quarterly  assessments.  PMO  officials  stated  that  they  did  not  use  the  assessment 
logs  or  perform  the  quarterly  assessments  because  their  constant  interaction  with 
Accenture  through  meetings  and  briefings  provided  adequate  information  to  assess 
contractor  performance. 

We  determined  that  collectively,  the  actual  ad  hoc  procedures  and  methods  used  by 
PMO  to  document  contractor  performance  met  the  intent  of  the  performance  plan. 
The  ad  hoc  procedures  and  documentation  methods  included  the  following: 

•  Milestone  Billing  -  Air  Force-developed  tracker  that  established  target 
monthly  tasks,  milestone  dates,  and  billing  amounts  incorporated  into 
the  contract.  The  acceptance/rejection  of  tasks  was  unofficially  tracked 
directly  on  the  milestone  billing  sheets  by  a  PMO  official. 

•  Program  Management  Review  -  Air  Force-developed  briefing  that  provided 
an  overview  of  the  program,  contract  status,  schedule,  financial  review, 
and  IPT  status. 
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•  Integrated  Master  Schedule  -  Contractor-developed  monthly  deliverables 
that  defined  tasks,  work  products,  milestones,  and  resource  requirements 
for  program  execution.  PMO  officials  maintained  copies  of  feedback 
that  they  provided  to  Accenture  identifying  critical,  substantial,  and 
administrative  issues. 

•  Incentive  Payment  Assessment  Documentation  -  The  incentive 
payment  assessment  documentation  tracked  contract  deliverables, 
performance  requirements,  due  dates,  acceptance  or  rejection,  and  level 
of  performance.  PMO  officials  adhered  to  established  assessment 
periods  when  determining  the  amounts  earned  by  Accenture  for 
incentive  payments. 

•  Monthly  Status  Report  -  Air  Force-developed  report  that  provided  an 
update  on  the  acquisition  and  execution  activities,  number  of  deficiency 
reports  worked,  Integrated  Master  Schedule  changes  since  the  last 
reporting  period,  and  contract  status. 

•  DEAMS  Program  Metrics  -  Air  Force-developed  briefing  that  provided 
an  update  on  user  metrics,  development  delivery  schedule,  operational 
assessment  readiness,  system  performance,  funding,  deployment  plan, 
and  deficiency  reports. 

•  Contracting  Officer  Pre-Approval  Process  -  The  contracting  officer 
established  procedures  requiring  prior  approval  before  incurring  travel 
and  other  direct  costs. 

PMO  officials  provided  evidence  that  they  used  and  unofficially  maintained  the 
ad  hoc  procedures  and  documentation  methods  listed  above. 

The  following  table  summarizes  the  performance  objectives  outlined  in  the 
performance  plan  and  compares  the  performance  plan  documentation  requirements 
with  the  actual  ad  hoc  procedures  and  documentation  methods  used  to 
document  surveillance. 
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Table.  Comparison  of  Performance  Plan  Requirements  and  Actual  Ad  Hoc  Procedures  and 
Documentation  Methods  Used  for  the  SI  Contract 


Performance  Objectives 
(Work  Requiring  Surveillance) 

Plan  Requirements 
for  Documenting 
Surveillance 

Actual  Method 
of  Documenting 
Surveillance 

Timeliness:  Deliverables  delivered  by 
the  date  specified  in  the  Contract  Data 
Requirements  List  (CDRL). 

SS-1:  CDRL  Delivery  Log 

•  Milestone  Billing 

•  Program  Management 
Review 

Quality  of  Service:  Deliverables  meet 
the  requirements  of  the  content  stated 
in  the  CDRL. 

SS-2:  Deliverable 
Acceptance  Log 

•  Milestone  Billing 

•  Incentive  Payment 
Assessment 
Documentation 

Timeliness:  Deficiency  Reports  fielded 
within  required  timelines. 

SS-3:  Deficiency  Report 
Timeline  Tracking  Log 

Integrated  Master 

Schedule 

Quality  of  Service:  Deficiency  Reports  are 
implemented  in  production  error-free. 

SS-4:  Requirements 
Traceability  Matrix 
Implementation 

Tracking  Chart 

•  Monthly  Status  Report 

•  DEAMS  Program 

Metric 

Timeliness:  Complete  the  release  by  the 
PMO  approved  Requirements  Operability 
Testing  Readiness  Review  schedule. 

SS-5:  Release 

Tracking  Table 

Milestone  Billing 

Cost  Control:  Meet  the  approved 
Contracting  Officers  Cost. 

SS-6  &  SS-7:  Total 

Contract  Cost  Tracking, 
Total  Contract  Cost  Time, 

&  Material  Tracking  Logs 

Contracting  Officer 
Pre-Approval  Process 

Timeliness:  Level  II  Help  Desk  Response 

SS-8:  Level  II  Help  Desk 
Tracking  Log 

Milestone  Billing 

The  actual  methods  of  documenting  surveillance  did  capture  contractor  performance 
results  that  addressed  the  performance  objectives  outlined  in  the  performance  plan. 
For  example,  one  of  the  performance  objectives  included  in  the  performance  plan  was 
to  assess  timeliness  and  determine  if  deliverables  were  delivered  by  the  date  specified 
within  the  Contract  Data  Requirements  List  (CDRL).  The  performance  plan  established 
performance  as  acceptable  when  98  percent  of  critical  deliverables  were  received  on 
time.  The  performance  plan  also  required  that  these  surveillance  results  be  captured 
on  a  CDRL  Delivery  Log.  Although  PMO  officials  did  not  use  the  CDRL  Delivery  Log 
to  capture  the  surveillance  results,  the  milestone  billing  tracking  spreadsheets  did 
capture  the  timeliness  performance  objective,  acceptance  criteria,  corresponding  CDRL, 
and  PMO  notes  documenting  surveillance  results. 


For  the  IV&V  contract,  a  contracting  officer  approved  the  performance  plan  on 
September  22,  2009.  The  performance  plan  identified  the  contract  elements  that 
required  surveillance  and  the  methods  used  to  provide  surveillance  as  required  by 
FAR  Subpart  46.4,  “Government  Contract  Quality  Assurance."  Specifically,  the 
performance  plan  outlined  the  performance  objectives,  performance  thresholds, 
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and  surveillance  methods.  In  addition,  the  performance  plan  established  the 
IV&V  IPT’s  responsibilities  for  monitoring  and  assessing  Ryan  Consulting’s 
performance,  which  included  preparing  and  maintaining  key  performance  data  such 
as  performance  logs,  performance  reports  and  contract  deliverables,  and  distributing 
this  information  quarterly  to  the  IV&V  IPT  members.  Based  on  the  content  of  the 
performance  plan,  we  determined  that  the  performance  plan  met  the  intent  of  a  QASP. 

The  actual  methods  of  documenting  surveillance  did  capture  contractor  performance 
results  that  addressed  the  performance  objectives.  One  of  the  performance  objectives 
included  in  the  performance  plan  was  for  Ryan  Consulting  to  provide  timely  and 
high-quality  project  status  reports.  PMO  officials  stated  that  they  met  weekly  with 
Ryan  Consulting,  provided  verbal  feedback  on  contractor  deliverables,  and  relied  on 
weekly  review  meeting  minutes  that  Ryan  Consulting  prepared  to  summarize 
discussions  and  details  of  performance  and  accomplishments.  PMO  officials  also 
maintained  the  weekly  review  meeting  minutes,  providing  evidence  that  PMO  officials 
and  Ryan  Consulting  discussed  current  and  future  tasks,  formal  and  informal 
deliverables,  and  milestones.  PMO  officials  also  used  the  weekly  review  meeting 
minutes  as  support  for  the  Contractor  Performance  Assessment  Report  (CPAR). 
PMO  officials  stated  that  they  did  not  identify  performance  issues  with 
Ryan  Consulting,  which  is  also  noted  in  the  CPAR.  PMO  officials  assessed  the 
quality  of  services  provided,  the  schedule,  and  other  performance  categories  and 
documented  their  assessment  in  the  CPAR.  PMO  officials  rated  Ryan  Consulting’s 
performance  as  "Exceptional"  from  September  30,  2009,  through  January  31,  2011, 
and  from  February  1,  2011,  through  January  31,  2012.  Although  PMO  officials  did 
not  prepare  and  maintain  performance  logs  and  reports  to  capture  the  surveillance 
results,  the  weekly  review  meeting  minutes  did  capture  the  timeliness  of  deliverables 
and  the  corresponding  CDRL.  The  CPARs  also  provided  evidence  of  surveillance 
results  and  feedback  delivered  to  Ryan  Consulting. 

Review  of  Internal  Controls 

DoD  Instruction  5010.40,  "Managers’  Internal  Control  Program  Procedures," 
May  30,  2013,  requires  DoD  organizations  to  implement  a  comprehensive  system  of 
internal  controls  that  provides  reasonable  assurance  that  programs  are  operating 
as  intended  and  to  evaluate  the  effectiveness  of  the  controls.  We  found  that  internal 
controls  over  the  SI  and  IV&V  contracts  were  generally  effective  as  they  related  to 
the  audit  objective. 
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Scope  and  Methodology 

We  conducted  this  performance  audit  from  February  2012  through  October  2013 
in  accordance  with  generally  accepted  government  auditing  standards.  Those 
standards  require  that  we  plan  and  perform  the  audit  to  obtain  sufficient, 
appropriate  evidence  to  provide  a  reasonable  basis  for  our  findings  and  conclusions 
based  on  our  audit  objectives.  We  believe  that  the  evidence  obtained  provides  a 
reasonable  basis  for  our  conclusions  based  on  our  audit  objectives. 

Our  announced  objective  was  to  determine  whether  DoD  and  Air  Force  officials 
administered  contracts  supporting  the  DEAMS  in  an  effective  and  efficient  manner. 
Specifically,  we  planned  to  determine  whether  Government  oversight  of  contractor 
performance  was  adequate,  quality  assurance  plans  were  adequately  implemented, 
contracts  were  funded  in  accordance  with  applicable  regulations,  and  modifications 
that  increased  costs  were  properly  supported.  During  the  planning  phase,  the 
FMO  provided  a  list  of  contracts  to  the  audit  team  in  April  2012.  The  PMO  office 
also  provided  a  list  of  contracts  to  the  audit  team  in  May  2012.  Both  lists  included 
contracts  awarded  on  behalf  of  the  FMO  and  PMO  offices.  We  compared  the  two  lists 
and  identified  duplicate  contract  numbers  on  both  lists.  We  reconciled  the  two  lists 
by  removing  the  duplicate  contracts,  which  resulted  in  a  universe  of  26  contracts, 
valued  at  approximately  $266  million.  Subsequently,  we  revised  our  specific  objectives 
to  focus  primarily  on  determining  whether  PMO  provided  adequate  oversight  of 
contractor  performance  and  implemented  quality  assurance  plans.  As  a  result,  we  did 
not  address  the  overall  objective  related  to  DoD  and  Air  Force  officials’  administration 
of  contracts  or  the  remaining  specific  objectives  related  to  contract  funding  and 
contract  modification. 

We  reviewed  two  FMO  contracts  and  two  PMO  contracts.  General  Services 
Administration  (GSA)  personnel  administered  the  two  FMO  contracts  reviewed.  We 
visited  the  GSA  Office  of  Assisted  Acquisition  Services  within  the  Federal  Acquisition 
Service  and  interviewed  personnel  to  gain  an  understanding  of  their  roles  in 
the  administration  of  the  two  FMO  contracts.  We  determined  that  GSA  was  not 
involved  with  onsite  contractor  performance  monitoring  for  the  two  FMO  contracts, 
but  relied  on  contractor  performance  monitoring  information  provided  by  FMO. 
Based  on  the  results  of  the  work  performed  during  the  planning  phase,  we  removed 
the  FMO  contracts  from  the  scope  of  the  audit.  At  the  time  of  our  site  visits,  the 
Electronic  Services  Group/Contracting  Branch  (ELSG/PK)  (formerly  known  as 
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Developing  and  Fielding  Systems  Group/Contract  Division)  was  the  contracting 
office  providing  contract  administration  for  the  two  PMO  contracts.  According  to 
Defense  Contract  Management  Agency  (DCMA)  personnel,  DCMA  shared  contract 
administration  responsibilities  with  ELSG/PK  for  the  SI  contract.  We  interviewed 
DCMA  personnel  to  gain  an  understanding  of  their  roles  and  responsibilities  for 
contract  administration.  DCMA  personnel  stated  they  provided  contract 
administration,  but  relied  on  contractor  performance  information  provided  by  the 
primary  contracting  officer  at  PMO.  Based  on  the  revised  scope  and  objectives  of  the 
audit,  we  did  not  perform  additional  work  related  to  DCMA  administrative  roles. 

We  visited  the  PMO  at  Wright-Patterson  Air  Force  Base  to  determine  if  the  SI  and 
IV&V  contracts  were  administered  in  an  effective  and  efficient  manner.  During  site 
visits,  we  interviewed  the  SI  primary  contracting  officer,  SI  and  IV&V  CORs,  and 
other  PMO  officials  to  gain  an  understanding  of  the  COR  delegation  and  training 
requirements,  contract  oversight  of  SI  and  IV&V  contractors,  and  their  methodology 
of  assessing  and  documenting  contractor  performance. 

We  reviewed  contract  documentation  from  2004  through  2012,  including  the 
SI  contract  FA8770-06-F-8001  and  the  IV&V  contract  FA8770-09-C-0064.  These 
contracts  had  an  approximate  total  value  of  $109  million.  We  also  reviewed 
COR  designation  letters,  nomination  letters,  training  certificates,  and  other 
documentation  to  support  contractor  oversight  for  the  SI  and  IV&V  contracts.  We 
determined  whether  contract  oversight  was  effective,  whether  QASPs  were  prepared, 
and  whether  contractor  performance  documentation  was  prepared  and  maintained. 

We  reviewed  Federal,  DoD,  and  Air  Force  contracting  guidance.  Specifically,  we 
reviewed  FAR  subpart  1.6,  FAR  subpart  46.4,  and  Defense  Federal  Acquisition 
Regulation  Supplement  Subpart  246.4,  "Government  Contract  Quality  Assurance." 
We  also  reviewed  the  Office  of  the  Under  Secretary  of  Defense  for  Acquisition, 
Technology,  and  Logistics  Memorandum,  "DoD  Standard  for  Certification  of  COR  for 
Service  Acquisitions,”  March  29,  2010,  and  Air  Force  Federal  Acquisition  Regulation 
Supplement  Mandatory  Procedures. 
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Use  of  Computer-Processed  Data 


We  did  not  rely  on  computer-processed  data  in  developing  our  findings  and  conclusions. 


Prior  Coverage 


No  prior  audit  coverage  has  been  conducted  on  PMO  oversight  of  DEAMS  contracts 
during  the  last  5  years. 

You  can  obtain  information  about  the  Department  of  Defense  Office  of  Inspector 
General  from  DoD  Directive  5106.01,  "Inspector  General  of  the  Department  of  Defense 
(IG  DoD),"  April  20,  2012;  DoD  Instruction  7600.02,  "Audit  Policies,”  April  27,  2007; 
and  DoD  Instruction  7050.03,  "Office  of  the  Inspector  General  of  the  Department 
of  Defense  Access  to  Records  and  Information,"  March  22,  2013.  Our  website  is 
www.dodig.mil. 

We  appreciate  the  courtesies  extended  to  the  staff.  If  you  have  any  questions, 
please  contact  me  at  (703)  601-5945  (DSN  329-5945). 


Lorin  T.  Venable,  CPA 
Assistant  Inspector  General 
Financial  Management  and  Reporting 
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Whistleblower  Protection 

U.S.  Department  of  Defense 


The  Whistleblower  Protection  Enhancement  Act  of  2012  requires 
the  Inspector  General  to  designate  a  Whistleblower  Protection 
Ombudsman  to  educate  agency  employees  about  prohibitions  on 
retaliation,  and  rights  and  remedies  against  retaliation  for  protected 
disclosures.  The  designated  ombudsman  is  the  DoD  IG  Director  for 
Whistleblowing  &  Transparency.  For  more  information  on  your  rights 
and  remedies  against  retaliation,  go  to  the  Whistleblower  webpage  at 
www.dodig.mil/programs/whistleblower. 


For  more  information  about  DoD  IG 
reports  or  activities,  please  contact  us: 

Congressional  Liaison 

Congressional@dodig.mil;  703.604.8324 

DoD  Hotline 

1.800.424.9098 

Media  Contact 

Public.Affairs@dodig.mil;  703.604.8324 

Monthly  Update 

dodigconnect-request@listserve.com 

Reports  Mailing  List 

dodig_report-request@listserve.com 

Twitter 

twitter.com/DoD_IG 


